{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Learning Goals\n",
    "\n",
    "## Optimizing Foundation Models with Parameter-Efficient Fine-Tuning (PEFT)\n",
    "\n",
    "This notebook aims to demonstrate how to adapt or customize foundation models to improve performance on specific tasks using NeMo 2.0.\n",
    "\n",
    "This optimization process is known as fine-tuning, which involves adjusting the weights of a pre-trained foundation model with custom data.\n",
    "\n",
    "Considering that foundation models can be significantly large, a variant of fine-tuning has gained traction recently known as PEFT. PEFT encompasses several methods, including P-Tuning, LoRA, Adapters, IA3, etc. NeMo 2.0 currently supports [Low-Rank Adaptation (LoRA)](https://arxiv.org/pdf/2106.09685) method.\n",
    "\n",
    "NeMo 2.0 introduces Python-based configurations, PyTorch Lightning’s modular abstractions, and NeMo-Run for scaling experiments across multiple GPUs. In this notebook, we will use NeMo-Run to streamline the configuration and execution of our experiments.\n",
    "\n",
    "## Data\n",
    "This notebook uses the SQuAD dataset. For more details about the data, refer to [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://arxiv.org/abs/1606.05250)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "## Step 1. Import the Hugging Face Checkpoint\n",
    "We use the `llm.import_ckpt` API to download the specified model using the \"hf://<huggingface_model_id>\" URL format. It will then convert the model into NeMo 2.0 format. For all model supported in NeMo 2.0, refer to [Large Language Models](https://docs.nvidia.com/nemo-framework/user-guide/24.09/llms/index.html#large-language-models) section of NeMo Framework User Guide."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import nemo_run as run\n",
    "from nemo import lightning as nl\n",
    "from nemo.collections import llm\n",
    "from megatron.core.optimizer import OptimizerConfig\n",
    "from nemo.collections.llm.peft.lora import LoRA\n",
    "import torch\n",
    "import lightning.pytorch as pl\n",
    "from pathlib import Path\n",
    "from nemo.collections.llm.recipes.precision.mixed_precision import bf16_mixed\n",
    "\n",
    "\n",
    "# llm.import_ckpt is the nemo2 API for converting Hugging Face checkpoint to NeMo format\n",
    "# example usage:\n",
    "# llm.import_ckpt(model=llm.llama3_8b.model(), source=\"hf://meta-llama/Meta-Llama-3-8B\")\n",
    "#\n",
    "# We use run.Partial to configure this function\n",
    "def configure_checkpoint_conversion():\n",
    "    return run.Partial(\n",
    "        llm.import_ckpt,\n",
    "        model=llm.llama3_8b.model(),\n",
    "        source=\"hf://meta-llama/Meta-Llama-3-8B\",\n",
    "        overwrite=False,\n",
    "    )\n",
    "\n",
    "# configure your function\n",
    "import_ckpt = configure_checkpoint_conversion()\n",
    "# define your executor\n",
    "local_executor = run.LocalExecutor()\n",
    "\n",
    "# run your experiment\n",
    "run.run(import_ckpt, executor=local_executor)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 2. Prepare the Data\n",
    "\n",
    "We will be using SQuAD for this notebook. NeMo 2.0 already provides a `SquadDataModule`. Example usage:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "def squad() -> run.Config[pl.LightningDataModule]:\n",
    "    return run.Config(llm.SquadDataModule, seq_length=2048, micro_batch_size=1, global_batch_size=8, num_workers=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To learn how to use your own data to create a custom `DataModule` for performing PEFT, refer to [NeMo 2.0 SFT notebook](./nemo2-sft.ipynb)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 3.1: Configure PEFT with NeMo 2.0 API and NeMo-Run\n",
    "\n",
    "The following Python script utilizes the NeMo 2.0 API to perform PEFT. In this script, we are configuring the following components for training. These components are similar between SFT and PEFT. SFT and PEFT both use `llm.finetune` API. To switch from SFT to PEFT, you just need to add `peft` with the LoRA adapter to the API parameter.\n",
    "\n",
    "### Configure the Trainer\n",
    "The NeMo 2.0 Trainer works similarly to the PyTorch Lightning trainer.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "def trainer() -> run.Config[nl.Trainer]:\n",
    "    strategy = run.Config(\n",
    "        nl.MegatronStrategy,\n",
    "        tensor_model_parallel_size=1\n",
    "    )\n",
    "    trainer = run.Config(\n",
    "        nl.Trainer,\n",
    "        devices=1,\n",
    "        max_steps=20,\n",
    "        accelerator=\"gpu\",\n",
    "        strategy=strategy,\n",
    "        plugins=bf16_mixed(),\n",
    "        log_every_n_steps=1,\n",
    "        limit_val_batches=2,\n",
    "        val_check_interval=2,\n",
    "        num_sanity_val_steps=0,\n",
    "    )\n",
    "    return trainer\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Configure the Logger\n",
    "Configure your training steps, output directories and logging through `NeMoLogger`. In the following example, the experiment output will be saved at `./results/nemo2_peft`.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "def logger() -> run.Config[nl.NeMoLogger]:\n",
    "    ckpt = run.Config(\n",
    "        nl.ModelCheckpoint,\n",
    "        save_last=True,\n",
    "        every_n_train_steps=10,\n",
    "        monitor=\"reduced_train_loss\",\n",
    "        save_top_k=1,\n",
    "        save_on_train_epoch_end=True,\n",
    "        save_optim_on_train_end=True,\n",
    "    )\n",
    "\n",
    "    return run.Config(\n",
    "        nl.NeMoLogger,\n",
    "        name=\"nemo2_peft\",\n",
    "        log_dir=\"./results\",\n",
    "        use_datetime_version=False,\n",
    "        ckpt=ckpt,\n",
    "        wandb=None\n",
    "    )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "\n",
    "### Configure the Optimizer\n",
    "In the following example, we will be using the distributed adam optimizer and pass in the optimizer configuration through `OptimizerConfig`: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "def adam() -> run.Config[nl.OptimizerModule]:\n",
    "    opt_cfg = run.Config(\n",
    "        OptimizerConfig,\n",
    "        optimizer=\"adam\",\n",
    "        lr=0.0001,\n",
    "        adam_beta2=0.98,\n",
    "        use_distributed_optimizer=True,\n",
    "        clip_grad=1.0,\n",
    "        bf16=True,\n",
    "    )\n",
    "    return run.Config(\n",
    "        nl.MegatronOptimizerModule,\n",
    "        config=opt_cfg\n",
    "    )\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Pass in the LoRA Adapter\n",
    "We need to pass in the LoRA adapter to our fine-tuning API to perform LoRA fine-tuning. We can configure the adapter as follows. The target module we support includes: `linear_qkv`, `linear_proj`, `linear_fc1` and `linear_fc2`. In the final script, we used the default configurations for LoRA (`llm.peft.LoRA()`), which will use the full list with `dim=32`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "def lora() -> run.Config[nl.pytorch.callbacks.PEFT]:\n",
    "    return run.Config(LoRA)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Configure the Base Model\n",
    "We will perform PEFT on top of Llama-3-8b, so we create a `LlamaModel` to pass to the NeMo 2.0 finetune API."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "def llama3_8b() -> run.Config[pl.LightningModule]:\n",
    "    return run.Config(llm.LlamaModel, config=run.Config(llm.Llama3Config8B))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Auto Resume\n",
    "In NeMo 2.0, we can directly pass in the Llama3-8b Hugging Face ID to start PEFT without manually converting it into the NeMo checkpoint, as required in NeMo 1.0."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "def resume() -> run.Config[nl.AutoResume]:\n",
    "    return run.Config(\n",
    "        nl.AutoResume,\n",
    "        restore_config=run.Config(nl.RestoreConfig,\n",
    "            path=\"nemo://meta-llama/Meta-Llama-3-8B\"\n",
    "        ),\n",
    "        resume_if_exists=True,\n",
    "    )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "### Configure the NeMo 2.0 finetune API\n",
    "Using all the components we created above, we can call the NeMo 2.0 finetune API. The python example usage is as below:\n",
    "```\n",
    "llm.finetune(\n",
    "    model=llama3_8b(),\n",
    "    data=squad(),\n",
    "    trainer=trainer(),\n",
    "    peft=lora(),\n",
    "    log=logger(),\n",
    "    optim=adam(),\n",
    "    resume=resume(),\n",
    ")\n",
    "```\n",
    "We configure the `llm.finetune` API as below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def configure_finetuning_recipe():\n",
    "    return run.Partial(\n",
    "        llm.finetune,\n",
    "        model=llama3_8b(),\n",
    "        trainer=trainer(),\n",
    "        data=squad(),\n",
    "        log=logger(),\n",
    "        peft=lora(),\n",
    "        optim=adam(),\n",
    "        resume=resume(),\n",
    "    )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 3.2: Run PEFT with NeMo 2.0 API and NeMo-Run\n",
    "\n",
    "We use `LocalExecutor` for executing our configured finetune function. For more details on the NeMo-Run executor, refer to [Execute NeMo Run](https://github.com/NVIDIA/NeMo-Run/blob/main/docs/source/guides/execution.md) of NeMo-Run Guides. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "def local_executor_torchrun(nodes: int = 1, devices: int = 1) -> run.LocalExecutor:\n",
    "    # Env vars for jobs are configured here\n",
    "    env_vars = {\n",
    "        \"TORCH_NCCL_AVOID_RECORD_STREAMS\": \"1\",\n",
    "        \"NCCL_NVLS_ENABLE\": \"0\",\n",
    "    }\n",
    "\n",
    "    executor = run.LocalExecutor(ntasks_per_node=devices, launcher=\"torchrun\", env_vars=env_vars)\n",
    "\n",
    "    return executor\n",
    "\n",
    "if __name__ == '__main__':\n",
    "    run.run(configure_finetuning_recipe(), executor=local_executor_torchrun())\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 4. Generate Results from Trained PEFT Checkpoints \n",
    "\n",
    "We use the `llm.generate` API in NeMo 2.0 to generate results from the trained PEFT checkpoint. Find your last saved checkpoint from your experiment dir: `results/nemo2_peft/checkpoints`. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "peft_ckpt_path=str(next((d for d in Path(\"./results/nemo2_peft/checkpoints/\").iterdir() if d.is_dir() and d.name.endswith(\"-last\")), None))\n",
    "print(\"We will load PEFT checkpoint from:\", peft_ckpt_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The SQuAD test set contains over 10,000 samples. For a quick demonstration, we will use the first 100 lines as an example input. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "head -n 100 /root/.cache/nemo/datasets/squad/test.jsonl > toy_testset.jsonl\n",
    "head -n 3 /root/.cache/nemo/datasets/squad/test.jsonl"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You should see something like:\n",
    "```\n",
    "{\"input\": \"Context: Super Bowl 50 was an American football game to determine the champion of the National Football League (NFL) for the 2015 season. The American Football Conference (AFC) champion Denver Broncos defeated the National Football Conference (NFC) champion Carolina Panthers 24\\u201310 to earn their third Super Bowl title. The game was played on February 7, 2016, at Levi's Stadium in the San Francisco Bay Area at Santa Clara, California. As this was the 50th Super Bowl, the league emphasized the \\\"golden anniversary\\\" with various gold-themed initiatives, as well as temporarily suspending the tradition of naming each Super Bowl game with Roman numerals (under which the game would have been known as \\\"Super Bowl L\\\"), so that the logo could prominently feature the Arabic numerals 50. Question: Which NFL team represented the AFC at Super Bowl 50? Answer:\", \"output\": \"Denver Broncos\", \"original_answers\": [\"Denver Broncos\", \"Denver Broncos\", \"Denver Broncos\"]}\n",
    "{\"input\": \"Context: Super Bowl 50 was an American football game to determine the champion of the National Football League (NFL) for the 2015 season. The American Football Conference (AFC) champion Denver Broncos defeated the National Football Conference (NFC) champion Carolina Panthers 24\\u201310 to earn their third Super Bowl title. The game was played on February 7, 2016, at Levi's Stadium in the San Francisco Bay Area at Santa Clara, California. As this was the 50th Super Bowl, the league emphasized the \\\"golden anniversary\\\" with various gold-themed initiatives, as well as temporarily suspending the tradition of naming each Super Bowl game with Roman numerals (under which the game would have been known as \\\"Super Bowl L\\\"), so that the logo could prominently feature the Arabic numerals 50. Question: Which NFL team represented the NFC at Super Bowl 50? Answer:\", \"output\": \"Carolina Panthers\", \"original_answers\": [\"Carolina Panthers\", \"Carolina Panthers\", \"Carolina Panthers\"]}\n",
    "{\"input\": \"Context: Super Bowl 50 was an American football game to determine the champion of the National Football League (NFL) for the 2015 season. The American Football Conference (AFC) champion Denver Broncos defeated the National Football Conference (NFC) champion Carolina Panthers 24\\u201310 to earn their third Super Bowl title. The game was played on February 7, 2016, at Levi's Stadium in the San Francisco Bay Area at Santa Clara, California. As this was the 50th Super Bowl, the league emphasized the \\\"golden anniversary\\\" with various gold-themed initiatives, as well as temporarily suspending the tradition of naming each Super Bowl game with Roman numerals (under which the game would have been known as \\\"Super Bowl L\\\"), so that the logo could prominently feature the Arabic numerals 50. Question: Where did Super Bowl 50 take place? Answer:\", \"output\": \"Santa Clara, California\", \"original_answers\": [\"Santa Clara, California\", \"Levi's Stadium\", \"Levi's Stadium in the San Francisco Bay Area at Santa Clara, California.\"]}\n",
    "\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will pass the string `toy_testset.jsonl` to the `input_dataset` parameter of `llm.generate`. To evaluate the entire test set, you can instead pass the SQuAD data module directly, using `input_dataset=squad()`. The input JSONL file should follow the format shown above, containing `input` and `output` fields (additional keys are optional)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from megatron.core.inference.common_inference_params import CommonInferenceParams\n",
    "\n",
    "\n",
    "def trainer() -> run.Config[nl.Trainer]:\n",
    "    strategy = run.Config(\n",
    "        nl.MegatronStrategy,\n",
    "        tensor_model_parallel_size=1\n",
    "    )\n",
    "    trainer = run.Config(\n",
    "        nl.Trainer,\n",
    "        accelerator=\"gpu\",\n",
    "        devices=1,\n",
    "        num_nodes=1,\n",
    "        strategy=strategy,\n",
    "        plugins=bf16_mixed(),\n",
    "    )\n",
    "    return trainer\n",
    "\n",
    "def configure_inference():\n",
    "    return run.Partial(\n",
    "        llm.generate,\n",
    "        path=str(peft_ckpt_path),\n",
    "        trainer=trainer(),\n",
    "        input_dataset=\"toy_testset.jsonl\",\n",
    "        inference_params=CommonInferenceParams(num_tokens_to_generate=20, top_k=1),\n",
    "        output_path=\"peft_prediction.jsonl\",\n",
    "    )\n",
    "\n",
    "\n",
    "def local_executor_torchrun(nodes: int = 1, devices: int = 1) -> run.LocalExecutor:\n",
    "    # Env vars for jobs are configured here\n",
    "    env_vars = {\n",
    "        \"TORCH_NCCL_AVOID_RECORD_STREAMS\": \"1\",\n",
    "        \"NCCL_NVLS_ENABLE\": \"0\",\n",
    "    }\n",
    "\n",
    "    executor = run.LocalExecutor(ntasks_per_node=devices, launcher=\"torchrun\", env_vars=env_vars)\n",
    "\n",
    "    return executor\n",
    "\n",
    "if __name__ == '__main__':\n",
    "    run.run(configure_inference(), executor=local_executor_torchrun())\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After the inference is complete, you will see results similar to the following:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "head -n 3 peft_prediction.jsonl"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You should see outputs similar to the following:\n",
    "```\n",
    "{\"input\": \"Context: Super Bowl 50 was an American football game to determine the champion of the National Football League (NFL) for the 2015 season. The American Football Conference (AFC) champion Denver Broncos defeated the National Football Conference (NFC) champion Carolina Panthers 24\\u201310 to earn their third Super Bowl title. The game was played on February 7, 2016, at Levi's Stadium in the San Francisco Bay Area at Santa Clara, California. As this was the 50th Super Bowl, the league emphasized the \\\"golden anniversary\\\" with various gold-themed initiatives, as well as temporarily suspending the tradition of naming each Super Bowl game with Roman numerals (under which the game would have been known as \\\"Super Bowl L\\\"), so that the logo could prominently feature the Arabic numerals 50. Question: Which NFL team represented the AFC at Super Bowl 50? Answer:\", \"original_answers\": [\"Denver Broncos\", \"Denver Broncos\", \"Denver Broncos\"], \"label\": \"Denver Broncos\", \"prediction\": \" Denver Broncos\"}\n",
    "{\"input\": \"Context: Super Bowl 50 was an American football game to determine the champion of the National Football League (NFL) for the 2015 season. The American Football Conference (AFC) champion Denver Broncos defeated the National Football Conference (NFC) champion Carolina Panthers 24\\u201310 to earn their third Super Bowl title. The game was played on February 7, 2016, at Levi's Stadium in the San Francisco Bay Area at Santa Clara, California. As this was the 50th Super Bowl, the league emphasized the \\\"golden anniversary\\\" with various gold-themed initiatives, as well as temporarily suspending the tradition of naming each Super Bowl game with Roman numerals (under which the game would have been known as \\\"Super Bowl L\\\"), so that the logo could prominently feature the Arabic numerals 50. Question: Which NFL team represented the NFC at Super Bowl 50? Answer:\", \"original_answers\": [\"Carolina Panthers\", \"Carolina Panthers\", \"Carolina Panthers\"], \"label\": \"Carolina Panthers\", \"prediction\": \" Carolina Panthers\"}\n",
    "{\"input\": \"Context: Super Bowl 50 was an American football game to determine the champion of the National Football League (NFL) for the 2015 season. The American Football Conference (AFC) champion Denver Broncos defeated the National Football Conference (NFC) champion Carolina Panthers 24\\u201310 to earn their third Super Bowl title. The game was played on February 7, 2016, at Levi's Stadium in the San Francisco Bay Area at Santa Clara, California. As this was the 50th Super Bowl, the league emphasized the \\\"golden anniversary\\\" with various gold-themed initiatives, as well as temporarily suspending the tradition of naming each Super Bowl game with Roman numerals (under which the game would have been known as \\\"Super Bowl L\\\"), so that the logo could prominently feature the Arabic numerals 50. Question: Where did Super Bowl 50 take place? Answer:\", \"original_answers\": [\"Santa Clara, California\", \"Levi's Stadium\", \"Levi's Stadium in the San Francisco Bay Area at Santa Clara, California.\"], \"label\": \"Santa Clara, California\", \"prediction\": \" Levi's Stadium\"}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 5. Calculate Evaluation Metrics\n",
    "\n",
    "We can evaluate the model's predictions by calculating the Exact Match (EM) and F1 scores.\n",
    "- Exact Match is a binary measure (0 or 1) checking if the model outputs match one of the\n",
    "ground truth answer exactly.\n",
    "- F1 score is the harmonic mean of precision and recall for the answer words.\n",
    "\n",
    "Below is a script that computes these metrics. The sample scores can be improved by training the model further and performing hyperparameter tuning. In this notebook, we only train for 20 steps.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "!python /opt/NeMo/scripts/metric_calculation/peft_metric_calc.py --pred_file peft_prediction.jsonl --label_field \"original_answers\" --pred_field \"prediction\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# NeMo Tools and Resources\n",
    "1. [NeMo GitHub repo](https://github.com/NVIDIA/NeMo)\n",
    "\n",
    "2. [NeMo-Run GitHub repo](https://github.com/NVIDIA/NeMo-Run/)\n",
    "\n",
    "3. NeMo Framework Container: `nvcr.io/nvidia/nemo:24.12`\n",
    "\n",
    "\n",
    "\n",
    "# Educational Resources\n",
    "1. Blog: [Mastering LLM Techniques: Customization](https://developer.nvidia.com/blog/selecting-large-language-model-customization-techniques/)\n",
    "\n",
    "2. Whitepaper: [LoRA: Low-Rank Adaptation of Large Language Models](https://arxiv.org/abs/2106.09685)\n",
    "\n",
    "3. [NeMo 2.0 Overview](https://docs.nvidia.com/nemo-framework/user-guide/latest/nemo-2.0/index.html)\n",
    "\n",
    "4. Blog: [Tune and Deploy LoRA LLMs with NVIDIA TensorRT-LLM](https://developer.nvidia.com/blog/tune-and-deploy-lora-llms-with-nvidia-tensorrt-llm/)\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
